Drawings reveal changes in object memory, but not spatial memory, across time.

Time has an immense influence on our memory. Truncated encoding leads to memory for only the 'gist' of an image, and long delays before recall result in generalized memories with few details. Here, we used crowdsourced scoring of hundreds of drawings made from memory after variable encoding (Experiment 1) and retentions of that memory (Experiment 2) to quantify what features of memory content change across time. We found that whereas some features of memory are highly dependent on time, such as the proportion of objects recalled from a scene and false recall for objects not in the original image, spatial memory was highly accurate and relatively independent of time. We also found that we could predict which objects were recalled across time based on the location, meaning, and saliency of the objects. The differential impact of time on object and spatial memory supports a separation of these memory systems.

Children develop adult-like visual sensitivity to image memorability by the age of 4.

Adults have been shown to consistently remember and forget certain images despite large individual differences, suggesting a population-wide sensitivity to an image's intrinsic memorability-a measure of how successfully an image is remembered. While a decade of research has focused on image memorability among adults, the developmental trajectory of these consistencies in memory is understudied. Here, we investigate by what age children gain adult-like sensitivity to the image memorability effect. We utilized data from Saragosa-Harris et al. (2021), where 137 children aged between 3 and 5 years old encoded animal-scene image pairs and then after a 5-min, 24-hr, or 1-week delay performed a cued recognition task for each scene target given its animal cue. We tested adults' memory of the same scene images using ResMem (Needell & Bainbridge, 2022), a pretrained deep neural network that predicts adult image memorability scores, and using an online behavioral continuous recognition task (N = 116). Results showed that ResMem predictions, as a proxy of adults' memory, predicted scene memory of children by the age of 4 and were the most predictive of children's memory across ages after a long, 1-week delay. Children at age 3 show nonadult-like consistent memory patterns, implying that the nonadult-like memory patterns were not due to poor memory performance. Instead, 3-year-olds may have consistently used certain visual memory strategies that become less optimal as they age. Our results suggest that adult-like sensitivity to image memorability emerges by the age of 4 through experience. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Memory for artwork is predictable.

Viewing art is often seen as a highly personal and subjective experience. However, are there universal factors that make a work of art memorable? We conducted three experiments, where we recorded online memory performance for 4,021 paintings from the Art Institute of Chicago, tested in-person memory after an unconstrained visit to the Art Institute, and obtained abstract attribute measures such as beauty and emotional valence for these pieces. Participants showed significant agreement in their memories both online and in-person, suggesting that pieces have an intrinsic "memorability" based solely on their visual properties that is predictive of memory in a naturalistic museum setting. Importantly, ResMem, a deep learning neural network designed to estimate image memorability, could significantly predict memory both online and in-person based on the images alone, and these predictions could not be explained by other low- or high-level attributes like color, content type, aesthetics, and emotion. A regression comprising ResMem and other stimulus factors could predict as much as half of the variance of in-person memory performance. Further, ResMem could predict the fame of a piece, despite having no cultural or historical knowledge. These results suggest that perceptual features of a painting play a major role in influencing its success, both in memory for a museum visit and in cultural memory over generations.

Neural similarity between overlapping events at learning differentially affects reinstatement across the cortex.

Episodic memory often involves high overlap between the actors, locations, and objects of everyday events. Under some circumstances, it may be beneficial to distinguish, or differentiate, neural representations of similar events to avoid interference at recall. Alternatively, forming overlapping representations of similar events, or integration, may aid recall by linking shared information between memories. It is currently unclear how the brain supports these seemingly conflicting functions of differentiation and integration. We used multivoxel pattern similarity analysis (MVPA) of fMRI data and neural-network analysis of visual similarity to examine how highly overlapping naturalistic events are encoded in patterns of cortical activity, and how the degree of differentiation versus integration at encoding affects later retrieval. Participants performed an episodic memory task in which they learned and recalled naturalistic video stimuli with high feature overlap. Visually similar videos were encoded in overlapping patterns of neural activity in temporal, parietal, and occipital regions, suggesting integration. We further found that encoding processes differentially predicted later reinstatement across the cortex. In visual processing regions in occipital cortex, greater differentiation at encoding predicted later reinstatement. Higher-level sensory processing regions in temporal and parietal lobes showed the opposite pattern, whereby highly integrated stimuli showed greater reinstatement. Moreover, integration in high-level sensory processing regions during encoding predicted greater accuracy and vividness at recall. These findings provide novel evidence that encoding-related differentiation and integration processes across the cortex have divergent effects on later recall of highly similar naturalistic events.

The features underlying the memorability of objects.

What makes certain images more memorable than others? While much of memory research has focused on participant effects, recent studies using a stimulus-centric perspective have sparked debate on the determinants of memory, including the roles of semantic and visual features and whether the most prototypical or atypical items are best remembered. Prior studies have typically relied on constrained stimulus sets, limiting a generalized view of the features underlying what we remember. Here, we collected more than 1 million memory ratings for a naturalistic dataset of 26,107 object images designed to comprehensively sample concrete objects. We establish a model of object features that is predictive of image memorability and examined whether memorability could be accounted for by the typicality of the objects. We find that semantic features exert a stronger influence than perceptual features on what we remember and that the relationship between memorability and typicality is more complex than a simple positive or negative association alone.

What moves us? The intrinsic memorability of dance.

Our most moving experiences, the ones that "stick," are hardly ever static but are dynamic, like a conversation, a gesture, or a dance. Previous work has shown robust memory for simple actions (e.g., jumping or turning), but it remains an open question how we remember more dynamic sequences of complex and expressive actions. Separately, with static images, previous work has found remarkable consistency in which images are remembered or forgotten across people-that is, an intrinsic "memorability"-but it is unclear whether semantically ambiguous and expressive actions might similarly be consistently remembered, despite the varying interpretations of what they could mean. How do we go from static memories to more memorable dynamic experiences? Using the test case of a rich and abstract series of actions from dance, we discover memorability as an intrinsic attribute of movement. Across genres, some movements were consistently remembered, regardless of the perceiver, and even regardless of the dancer. Among a comprehensive set of memory, movement, and aesthetic attributes, consistency in which movements people remembered was most predicted by subjective memorability, and importantly by both subjective (observer ratings) and objective (optical flow analysis) measures of the scale of motion, such that the less overall motion in a dance segment, the more memorable the movements tended to be. Importantly, we discover that memorability of a sequence is additive, where the memorability of individual snapshots and constituent moments ultimately contribute to the memorability of longer sequences. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Memorable beginnings, but forgettable endings: Intrinsic memorability alters our subjective experience of time.

Time is the fabric of experience - yet it is incredibly malleable in the mind of the observer: seeming to drag on, or fly right by at different moments. One of the most influential drivers of temporal distortions is attention, where heightened attention dilates subjective time. But an equally important feature of subjective experience involves not just the objects of attention, but also what information will naturally be remembered or forgotten, independent of attention (i.e. intrinsic image memorability). Here we test how memorability influences time perception. Observers viewed scenes in an oddball paradigm, where the last scene could be a forgettable "oddball" amidst memorable ones, or vice versa. Subjective time dilation occurred only for forgettable oddballs, but not memorable ones - demonstrating an oddball effect where the oddball did not differ in low-level visual features, image category, or even subjective memorability. But more importantly, these results emphasize how memory can interact with temporal experience: forgettable endings amidst memorable sequences dilate our experience of time.

Item memorability has no influence on value-based decisions.

While making decisions, we often rely on past experiences to guide our choices. However, not all experiences are remembered equally well, and some elements of an experience are more memorable than others. Thus, the intrinsic memorability of past experiences may bias our decisions. Here, we hypothesized that individuals would tend to choose more memorable options than less memorable ones. We investigated the effect of item memorability on choice in two experiments. First, using food images, we found that the same items were consistently remembered, and others consistently forgotten, across participants. However, contrary to our hypothesis, we found that participants did not prefer or choose the more memorable over the less memorable items when choice options were matched for the individuals' valuation of the items. Second, we replicated these findings in an alternate stimulus domain, using words that described the same food items. These findings suggest that stimulus memorability does not play a significant role in determining choice based on subjective value.

Multidimensional memory topography in the medial parietal cortex identified from neuroimaging of thousands of daily memory videos.

Our memories form a tapestry of events, people, and places, woven across the decades of our lives. However, research has often been limited in assessing the nature of episodic memory by using artificial stimuli and short time scales. The explosion of social media enables new ways to examine the neural representations of naturalistic episodic memories, for features like the memory's age, location, memory strength, and emotions. We recruited 23 users of a video diary app ("1 s Everyday"), who had recorded 9266 daily memory videos spanning up to 7 years. During a 3 T fMRI scan, participants viewed 300 of their memory videos intermixed with 300 from another individual. We find that memory features are tightly interrelated, highlighting the need to test them in conjunction, and discover a multidimensional topography in medial parietal cortex, with subregions sensitive to a memory's age, strength, and the familiarity of the people and places involved.

The Visual Mandela Effect as Evidence for Shared and Specific False Memories Across People.

The Mandela effect is an Internet phenomenon describing shared and consistent false memories for specific icons in popular culture. The visual Mandela effect is a Mandela effect specific to visual icons (e.g., the Monopoly Man is falsely remembered as having a monocle) and has not yet been empirically quantified or tested. In Experiment 1 (N = 100 adults), we demonstrated that certain images from popular iconography elicit consistent, specific false memories. In Experiment 2 (N = 60 adults), using eye-tracking-like methods, we found no attentional or visual differences that drive this phenomenon. There is no clear difference in the natural visual experience of these images (Experiment 3), and these errors also occur spontaneously during recall (Experiment 4; N = 50 adults). These results demonstrate that there are certain images for which people consistently make the same false-memory error, despite the majority of visual experience being the canonical image.

The extent and specificity of visual exploration determines the formation of recollected memories in complex scenes.

Our visual memories of complex scenes often appear as robust, detailed records of the past. Several studies have demonstrated that active exploration with eye movements improves recognition memory for scenes, but it is unclear whether this improvement is due to stronger feelings of familiarity or more detailed recollection. We related the extent and specificity of fixation patterns at encoding and retrieval to different recognition decisions in an incidental memory paradigm. After incidental encoding of 240 real-world scene photographs, participants (N = 44) answered a surprise memory test by reporting whether an image was new, remembered (indicating recollection), or just known to be old (indicating familiarity). To assess the specificity of their visual memories, we devised a novel report procedure in which participants selected the scene region that they specifically recollected, that appeared most familiar, or that was particularly new to them. At encoding, when considering the entire scene,subsequently recollected compared to familiar or forgotten scenes showed a larger number of fixations that were more broadly distributed, suggesting that more extensive visual exploration determines stronger and more detailed memories. However, when considering only the memory-relevant image areas, fixations were more dense and more clustered for subsequently recollected compared to subsequently familiar scenes. At retrieval, the extent of visual exploration was more restricted for recollected compared to new or forgotten scenes, with a smaller number of fixations. Importantly, fixation density and clustering was greater in memory-relevant areas for recollected versus familiar or falsely recognized images. Our findings suggest that more extensive visual exploration across the entire scene, with a subset of more focal and dense fixations in specific image areas, leads to increased potential for recollecting specific image aspects.

Predicting visual memory across images and within individuals.

We only remember a fraction of what we see-including images that are highly memorable and those that we encounter during highly attentive states. However, most models of human memory disregard both an image's memorability and an individual's fluctuating attentional states. Here, we build the first model of memory synthesizing these two disparate factors to predict subsequent image recognition. We combine memorability scores of 1100 images (Experiment 1, n = 706) and attentional state indexed by response time on a continuous performance task (Experiments 2 and 3, n = 57 total). Image memorability and sustained attentional state explained significant variance in image memory, and a joint model of memory including both factors outperformed models including either factor alone. Furthermore, models including both factors successfully predicted memory in an out-of-sample group. Thus, building models based on individual- and image-specific factors allows for directed forecasting of our memories. SIGNIFICANCE STATEMENT: Although memory is a fundamental cognitive process, much of the time memory failures cannot be predicted until it is too late. However, in this study, we show that much of memory is surprisingly pre-determined ahead of time, by factors shared across the population and highly specific to each individual. Specifically, we build a new multidimensional model that predicts memory based just on the images a person sees and when they see them. This research synthesizes findings from disparate domains ranging from computer vision, attention, and memory into a predictive model. These findings have resounding implications for domains such as education, business, and marketing, where it is a top priority to predict (and even manipulate) what information people will remember.

A tutorial on capturing mental representations through drawing and crowd-sourced scoring.

When we draw, we are depicting a rich mental representation reflecting a memory, percept, schema, imagination, or feeling. In spite of the abundance of data created by drawings, drawings are rarely used as an output measure in the field of psychology, due to concerns about their large variance and their difficulty of quantification. However, recent work leveraging pen-tracking, computer vision, and online crowd-sourcing has revealed new ways to capture and objectively quantify drawings, to answer a wide range of questions across fields of psychology. Here, I present a tutorial on modern methods for drawing experiments, ranging from how to quantify pen-and-paper type studies, up to how to administer a fully closed-loop online experiment. I go through the concrete steps of designing a drawing experiment, recording drawings, and objectively quantifying them through online crowd-sourcing and computer vision methods. Included with this tutorial are code examples at different levels of complexity and tutorials designed to teach basic lessons about web architecture and be useful regardless of skill level. I also discuss key methodological points of consideration, and provide a series of potential jumping points for drawing studies across fields in psychology. I hope this tutorial will arm more researchers with the skills to capture these naturalistic snapshots of a mental image.

Highly similar and competing visual scenes lead to diminished object but not spatial detail in memory drawings.

Drawings of scenes made from memory can be highly detailed and spatially accurate, with little information not found in the observed stimuli. While prior work has focused on studying memory for distinct scenes, less is known about the specific detail recalled when episodes are highly similar and competing. Here, participants (N = 30) were asked to study and recall eight complex scene images using a drawing task. Importantly, four of these images were exemplars of different scene categories, while the other four images were from the same scene category. The resulting 213 drawings were judged by 1764 online scorers for a comprehensive set of measures, including scene and object diagnosticity, spatial information, and fixation and pen movement behaviour. We observed that competition in memory resulted in diminished object detail, with drawings and objects that were less diagnostic of their original image. However, repeated exemplars of a category did not result in differences in spatial memory accuracy, and there were no differences in fixations during study or pen movements during recall. These results reveal that while drawings for distinct categories of scenes can be highly detailed and accurate, drawings for scenes from repeated categories, creating competition in memory, show reduced object detail.

Content-specific vulnerability of recent episodic memories in Alzheimer's disease.

Endel Tulving's episodic memory framework emphasizes the multifaceted re-experiencing of personal events. Indeed, decades of research focused on the experiential nature of episodic memories, usually treating recent episodic memory as a coherent experiential quality. However, recent insights into the functional architecture of the medial temporal lobe show that different types of mnemonic information are segregated into distinct neural pathways in brain circuits empirically associated with episodic memory. Moreover, recent memories do not fade as a whole under conditions of progressive neurodegeneration in these brain circuits, notably in Alzheimer's disease. Instead, certain memory content seem particularly vulnerable from the moment of their encoding while other content can remain memorable consistently across individuals and contexts. We propose that these observations are related to the content-specific functional architecture of the medial temporal lobe and consequently to a content-specific impairment of memory at different stages of the neurodegeneration. To develop Endel Tulving's inspirational legacy further and to advance our understanding of how memory function is affected by neurodegenerative conditions such as Alzheimer's disease, we postulate that it is compelling to focus on the representational content of recent episodic memories.

Disrupted object-scene semantics boost scene recall but diminish object recall in drawings from memory.

Humans are highly sensitive to the statistical relationships between features and objects within visual scenes. Inconsistent objects within scenes (e.g., a mailbox in a bedroom) instantly jump out to us and are known to catch our attention. However, it is debated whether such semantic inconsistencies result in boosted memory for the scene, impaired memory, or have no influence on memory. Here, we examined the relationship of scene-object consistencies on memory representations measured through drawings made during recall. Participants (N = 30) were eye-tracked while studying 12 real-world scene images with an added object that was either semantically consistent or inconsistent. After a 6-minute distractor task, they drew the scenes from memory while pen movements were tracked electronically. Online scorers (N = 1,725) rated each drawing for diagnosticity, object detail, spatial detail, and memory errors. Inconsistent scenes were recalled more frequently, but contained less object detail. Further, inconsistent objects elicited more errors reflecting looser memory binding (e.g., migration across images). These results point to a dual effect in memory of boosted global (scene) but diminished local (object) information. Finally, we observed that participants fixate longest on inconsistent objects, but these fixations during study were not correlated with recall performance, time, or drawing order. In sum, these results show a nuanced effect of scene inconsistencies on memory detail during recall.

Quantifying aphantasia through drawing: Those without visual imagery show deficits in object but not spatial memory.

Congenital aphantasia is a recently characterized variation of experience defined by the inability to form voluntary visual imagery, in individuals who are otherwise high performing. Because of this specific deficit to visual imagery, individuals with aphantasia serve as an ideal group for probing the nature of representations in visual memory, particularly the interplay of object, spatial, and symbolic information. Here, we conducted a large-scale online study of aphantasia and revealed a dissociation in object and spatial content in their memory representations. Sixty-one individuals with aphantasia and matched controls with typical imagery studied real-world scene images, and were asked to draw them from memory, and then later copy them during a matched perceptual condition. Drawings were objectively quantified by 2,795 online scorers for object and spatial details. Aphantasic participants recalled significantly fewer objects than controls, with less color in their drawings, and an increased reliance on verbal scaffolding. However, aphantasic participants showed high spatial accuracy equivalent to controls, and made significantly fewer memory errors. These differences between groups only manifested during recall, with no differences between groups during the matched perceptual condition. This object-specific memory impairment in individuals with aphantasia provides evidence for separate systems in memory that support object versus spatial information. The study also provides an important experimental validation for the existence of aphantasia as a variation in human imagery experience.

Distinct Representational Structure and Localization for Visual Encoding and Recall during Visual Imagery.

During memory recall and visual imagery, reinstatement is thought to occur as an echoing of the neural patterns during encoding. However, the precise information in these recall traces is relatively unknown, with previous work primarily investigating either broad distinctions or specific images, rarely bridging these levels of information. Using ultra-high-field (7T) functional magnetic resonance imaging with an item-based visual recall task, we conducted an in-depth comparison of encoding and recall along a spectrum of granularity, from coarse (scenes, objects) to mid (e.g., natural, manmade scenes) to fine (e.g., living room, cupcake) levels. In the scanner, participants viewed a trial-unique item, and after a distractor task, visually imagined the initial item. During encoding, we observed decodable information at all levels of granularity in category-selective visual cortex. In contrast, information during recall was primarily at the coarse level with fine-level information in some areas; there was no evidence of mid-level information. A closer look revealed segregation between voxels showing the strongest effects during encoding and those during recall, and peaks of encoding-recall similarity extended anterior to category-selective cortex. Collectively, these results suggest visual recall is not merely a reactivation of encoding patterns, displaying a different representational structure and localization from encoding, despite some overlap.

Reply to Intraub.

Intraub posits the existence of two separate processes in scene memory: one in which we automatically extrapolate the visual information in a scene beyond its boundaries (scene construction), and one in which we normalize our memories to either a schema or an average (normalization). She claims that scene construction will lead to transformations exclusively in the direction of boundary extension (BE), while normalization will produce bidirectional transformations of both BE and the opposite effect of boundary contraction (BC). Thus, because we observed both BE and BC in our study [1], our paradigm must be tapping into additional cognitive processes than just scene construction. However, our paper [1] questions this premise - if BE and BC are equally common using large, representative stimulus sets in the same tasks used previously [2,3], then perhaps prior studies primarily found unidirectional BE effects due to limited stimulus sampling, and there was no privileged link between boundary extension and scene construction in memory to begin with.

Memorability of words in arbitrary verbal associations modulates memory retrieval in the anterior temporal lobe.

Despite large individual differences in memory performance, people remember certain stimuli with overwhelming consistency. This phenomenon is referred to as the memorability of an individual item. However, it remains unknown whether memorability also affects our ability to retrieve associations between items. Here, using a paired-associates verbal memory task, we combine behavioural data, computational modelling and direct recordings from the human brain to examine how memorability influences associative memory retrieval. We find that certain words are correctly retrieved across participants irrespective of the cues used to initiate memory retrieval. These words, which share greater semantic similarity with other words, are more readily available during retrieval and lead to more intrusions when retrieval fails. Successful retrieval of these memorable items, relative to less memorable ones, results in faster reinstatement of neural activity in the anterior temporal lobe. Collectively, our data reveal how the brain prioritizes certain information to facilitate memory retrieval.